Belief-propagation algorithm and the Ising model on networks with arbitrary 

distributions of motifs 
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We generalize the belief-propagation algorithm to sparse random networks with arbitrary distribu- 
tions of motifs (triangles, loops, etc.). Each vertex in these networks belongs to a given set of motifs 
(generalization of the configuration model). These networks can be treated as sparse uncorrelated 
hypergraphs in which hyperedges represent motifs. Here a hypergraph is a generalization of a graph, 
where a hyperedge can connect any number of vertices. These uncorrelated hypergraphs are tree- 
like (hypertrees) , which crucially simplify the problem and allow us to apply the belief-propagation 
algorithm to these loopy networks with arbitrary motifs. As natural examples, we consider motifs 
in the form of finite loops and cliques. We apply the belief-propagation algorithm to the ferromag- 
netic Ising model on the resulting random networks. We obtain an exact solution of this model 
on networks with finite loops or cliques as motifs. We find an exact critical temperature of the 
ferromagnetic phase transition and demonstrate that with increasing the clustering coefficient and 
the loop size, the critical temperature increases compared to ordinary tree-like complex networks. 
Our solution also gives the birth point of the giant connected component in these loopy networks. 

PACS numbers: 05.10.-a, 05.40.-a, 05.50.+q, 87.18.Sn 



I. INTRODUCTION 

The belief-propagation algorithm is an effective nu- 
merical method for solving inference problems on sparse 
graphs. It was originally proposed by J. Pearl [1| for 
tree-like graphs. The belief-propagation algorithm was 
applied to study diverse systems in computer science, 
physics, and biology. Among its numerous applications 
are computer vision problems, decoding of high perfor- 
mance turbo codes and many others, see 0, Q. Empir- 
ically, it was found that this algorithm works surpris- 
ingly good even for graphs with loops. Yedidia et al. 
[H found that the belief-propagation algorithm actually 
coincides with the minimization of the Bethe free en- 
ergy. This discovery renewed interest in the Bethe-Peierls 
approximation and related methods p-Gj. In statistical 
physics the belief-propagation algorithm is equivalent to 
the so-called cavity method proposed by Mezard, Parisi, 
and Virasoro [8|. The belief-propagation algorithm is 
applicable to models with both discrete and continuous 
variables [9l-fllj|. Recently the belief-propagation algo- 
rithm was applied to diverse problems in random net- 
works: the disease sprea ding [12| . the calculation of the 
size of giant component [13], counting large loops in di- 
rected random uncorrelated networks [l4|, the graph bi- 
partitioning problem [l5| . the analysis of the contribu- 
tions of individual nodes and groups of nodes to the net- 
work structure [l6j , and networks of spiking neurons [l7| . 

These investigations showed that the belief- 
propagation algorithm is exact for a model on a 
graph with locally tree-like structure in the infinite 
size limit. However, only an approximate solution was 
obtained by use of this method for networks with short 
loops. Real networks have numerous short and large 
loops and other complex subgraphs or motifs which 



lead to essentially non-tree-like neighborhoods. That is 
why it is important to develop a method which takes 
into account finite loops and more complex motifs. 
Different ways were proposed recently to compute loop 
corrections to the Bethe approximation by use of the 
belief-propagation algorithm [TH, These methods, 

however, are exact only if a graph contains a single 
loop. Loop expansions were proposed in Refs. [20l l2lj|. 
However, they were not applied to complex network yet. 



Recently, Newman and Miller [22-24] independently 
introduced a model of random graphs with arbitrary dis- 
tributions of subgraphs or motifs. In this natural general- 
ization of the configuration model, each vertex belongs to 
a given set of motifs (e.g., vertex i is a member of Q^'(i) 
motifs of type 1, Q^ 2 \i) motifs of type 2, and so on), and 
apart of this constraint, the network is uniformly ran- 
dom. In the original configuration model, the sequence 
of vertex degrees Q(i) is fixed, where i = 1,2, ...,N. In 
this generalization, the sequence of generalized vertex de- 
grees Q^(i), Q^(i), ■•■ is given. For example, motifs can 
be triangles, loops, chains, cliques (fully connected sub- 
graphs), single edges which do not enter other motifs, 
and, in general, arbitrary finite clusters. The resulting 
networks can be treated as uncorrelated hypergraphs in 
which hyperedges represent motifs. In graph theory, a 
hypergraph is a generalization of a graph, where a hy- 
peredge can connect any number of vertices 25]. Be- 
cause of the complex motifs, the large sparse networks 
under consideration can have loops, clustering, and corre- 
lations, but the underlying hypergraphs are locally tree- 
like (hypertrees) and uncorrelated similarly to the orig- 
inal sparse configuration model. Our approach is based 
on the hypertree-like structure of these highly structured 
sparse networks. To demonstrate our approach, we apply 
the generalized belief-propagation algorithm to the fer- 
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romagnetic Ising model on the sparse networks in which 
motifs arc finite loops or cliques. We obtain an exact 
solution of the ferromagnetic Ising model and the birth 
point of the giant connected component in this kind of 
highly structured networks. Note that the Ising model 
on these networks is a more complex problem than the 
percolation problem because we must account for spin in- 
teractions between spins both inside and between motifs. 
We find an exact critical temperature of the ferromag- 
netic phase transition and demonstrate that finite loops 
increase the critical temperature in comparison to ordi- 
nary tree-like networks. 

II. ENSEMBLE OF RANDOM NETWORKS 
WITH MOTIFS 

Let us introduce a statistical ensemble of random net- 
works with a given distribution of motifs in which each 
vertex belongs to a given set of motifs and apart of this 
constraint, the networks are uniformly random. These 
networks can be treated as uncorrclatcd hypergraphs, in 
which motifs play a role of hyperedges, so that the num- 
ber of hyperdegrees are equal to the number of specific 
motifs attached to a vertex. In principal, one can choose 
any subgraph with an arbitrary number of vertices as a 
motif, see Fig. [T] In the present paper, for simplicity, we 
only consider simple motifs such as single edges, finite 
loops, and cliques. 

Let us first remind how one can describe a statisti- 
cal ensemble of random networks with the simplest mo- 
tifs, namely simple edges (see, for example, Refs. [t| I2H ] 
and references therein). We define the probability P2{a>ij) 
that an edge between vertices i and j is present (oy = 1) 
or absent (ay = 0), 

Man) = ^S( aiJ - 1) + (l - J^K) (1) 

where are entries of the symmetrical adjacency ma- 
trix, (Q2) = (Q) is the mean number of edges attached to 
a vertex. It is well known that the probability of the real- 
ization of the Erdos-Renyi graph with a given adjacency 
matrix a 13 , is the product 

N—i n 

G2({a tJ }) = J] J] P 2( aij ). (2) 

8=1 j=i+l 

The degree distribution is the Poisson distribution. In 
the configuration model, the probability of the realization 
of a given graph with a given sequence of degrees, Q2(l), 
Q 2 (2) ) Q 2 (3) ) ...,Q 2 (A0 = {Q 2 (i)},i s 

N N 

G 2 (K-}) = jUs{Q 2 (i)-J2 a ») Jlrt<Hi)- ( 3 ) 

»=1 j=l i<j 

The delta-function fixes the number of edges attached to 
vertex i. A is a normalization constant. The distribu- 
tion function of degrees is determined by the sequence 



{Qa(i)}, 

P 2 (Q 2 ) = ^£>(Q2-Q 2 (i)). (4) 

i 

The second simplest motif, triangle, plays the role of 
a hyperedge which interconnects a triple of vertices. Let 
us introduce the probability pz{dijk) that a hyperedge 
(triangle) between vertices i, j, and k is present or absent, 
i.e., dijk = 1 or a%jk = 0, respectively: 

P3(ciijk) = p5{a ijk - 1) + (1 - p)5(aijk), (5) 

where 



y (N-l)(N-2) y ' 

is the probability that vertices i, j and k form a triangle. 
(Q3) is the mean number of triangles attached to a ran- 
domly chosen vertex, are entries of the adjacency 
matrix of the hypergraph. This matrix is symmetrical 
with respect to permutations of the indices i,j and k, 

&ijk — &jki — Q>kij — ■ ■ • • 

For example, one can introduce the ensemble of the 
Erdos-Renyi hypergraphs. Given that the matrix ele- 
ments ciijk are independent and uncorrelated random pa- 
rameters, the probability of realization of a graph with a 
given adjacency matrix ay-fc, is the product of probabili- 
ties P3,{a,ijk) over different triples of vertices: 

N-2 N—l N 

G3({a ijk }) = n n n psm. en 

i=l j=i+l k=j+l 

This function describes the ensemble of the Erdos-Renyi 
hypergraphs with the Poisson degree distributions of the 
number of triangles attached to vertices, 

P 3 (Q3 )= «^! e -<Q3>. (8) 

In the configuration model, the probability of the re- 
alization of a given graph with a sequence of the number 
of triangles, Q 8 (l), Q 3 (2), Q 3 (3),. . . ,Q 3 (N) = {Q 3 (*)}» 
attached to vertices i = 1, 2, . . . , N is defined by an equa- 
tion, 

1 N 1 

G 3 ({a ijk }) = -jYl^{Q3(i) - 2 ^2 a ijk) II P3( a ijk)- 
i—1 j.k i<j<k 

_ (9) 

Here P3{atjk) is given by Eq. ([S]). The delta- function 
fixes the number of triangles attached to vertex i. A is 
a normalization constant. The distribution function of 
triangles is 

P 3(Q3) = ^I>W3-Q3«). (10) 
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total number of edges which overlap with triangles equals 



FIG. 1. (Color online) Different motifs can be attached to 
vertex i in a hypergraph: an ordinary edge a,*, a triangle, a 
square, and pentagon. There are also non-symmetric motifs 
consisting of four vertices and the clique of size 4. A mo- 
tif can have internal vertices (open circle) which have edges 
only inside this motif. Motifs form a locally tree-like hyper- 
graph where they play a role of hyperedges. Arrows represent 
incoming messages which arrive at vertex i from the motifs 
attached to this vertex. 



One can further generalize equations Eqs. (J5]) and 
© and introduce the probability pA(&ijki) that vertices 
i,j,k, and I form a hyperedge of size 4 (a clique or loop 
of size 4). In the case of loops, one can arrange these 
vertices in order of increasing vertex index. Then, for a 
given sequence of squares or cliques, {Q4(l), Qa{2), . . . }, 
attached to vertices i = 1,2,..., one can introduce 
the probability Gi({dijki}) of the realization of a given 
graph with a given sequence {Q^i)} similar to Eq. ©, 
and so on. The network ensemble of the configuration 
model with a given sequences of edges {Q2{i)}> triangles 
{Qs{i)}, squares {Q^i)} and other motifs is described 
by a product of the corresponding probabilities, 

G{{a>ij}, Wijk}, Wijki}, •••) = G2({aij})G 3 ({aijk})— 

(11) 

The average of a quantity K ({a^}, {a^k}, ...) over the 
network ensemble is 



(if) en =/ K May}, {dijk}, {Oijkl} — 



G({a,ij}, {dijk}, {dijki}, •■•) ddij Y[ ddijk JJdciijfc; 

i<j i<j<k i<j<k<l 



(12) 



where we integrate over all possible edges and hyper- 
edges. 

One can prove that the probability that different motifs 
have a common edge, i.e., they are overlapping, tends to 
zero in the infinite size limit N — ► oo. For example, the 



ij Qijk 



N(Q 2 ){Q 3 ) 
N - I 



(13) 



This number of double connections is finite in the limit 
N — > oo. Because the total number of edges is of the 
order of N, the fraction of double connections tends to 
zero in the thermodynamic limit. Thus, one can neglect 
overlapping between different motifs. Usingthe stan- 
dard methods in complex network theory |27H23 |. one 
can show that in the configuration model the number of 
finite loops formed by hyperedges is finite in the thermo- 
dynamic limit. Therefore, sparse random uncorrelated 
hypergraphs have a hypertree-like structure. 

Uncorrelated random hypergraphs are described by 
a distribution function P(Q 2 , Q3, Qa, ■ ■ ■ ) which is the 
probability that a randomly chosen vertex has Qi edges, 
Q3 triangles, Q4 squares, and other motifs attached to 
this vertex. In the case of uncorrelated motifs, we have 



P{Q2,Q3,Q4, 



Y[Pt{Qt). 



(14) 



Here, Pi(Qi) is the distribution function of loops of size 



(15) 



In this kind of network, the number of nearest neighbor- 
ing vertices of vertex i is equal to 

Q(i) = Q 2 (i) + 2Q 3 (i) + 2Q 4 (i) + . . . . (16) 

Therefore, the mean degree is 

1 N 

W)=vEW = E QP{Q*Q3,Q*,...). (17) 



»=i 



The local clustering coefficient C(i) of node i with de- 
gree Q(i), Eq. (|16l) . is determined by the number of tri- 
angles Qz{i) as follows, 



C(i) 



2Q 3 (») 



Q(i)(Q(i) - 1) 



(18) 



Therefore, C(i) is maximum if a network only consists 
of triangles. In this case, we have Q(i) = 2Q 3 (i) and 
therefore 



C{i) = 



1 



Q(i)-1 



(19) 



On the other hand, if a network only consists of ^-cliques 
with a given I (fully connected subgraphs of size £), then 
the local clustering coefficient of vertex i with Qi(i) at- 
tached cliques equals 



C(i) 



£-2 

Q(«)-i : 



(20) 
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where Q(i) = (£ — l)Qe(i) is degree of vertex i. If 
other motifs (edges, squares, and larger finite loops) are 
present in the network, then according to Eqs. (|18[) and 
(HO]) the local clustering coefficient C(i) is smaller than 
(£ - 2)/(Q(i) - 1). In Refs. jUSI, it was shown that 
properties of networks with "weak clustering", C(Q) ~ 
0(1/ Q), may be different from properties of networks 
with "strong clustering" when C(Q) decreases slower 
than 1/Q. The latter networks are out of the scope of 
the present paper. 



III. BELIEF-PROPAGATION ALGORITHM 



moment of spin i, 



(Si) 



SiPi(Si). 



(24) 



Si=±l 



A message fj,ji(Si) can be written in a general form, 

fijiiSi) = expiphjiSJ/ficosHphji)}, (25) 

Using this representation, we rewrite Eq. (|24|) in a phys- 
ically clear form, 



(Si) = tanh^ffi + a 3i h ji) ■ 



(26) 



Let us consider the Ising model on a sparse random 
network with arbitrary distributions of motifs. The 
Hamiltonian of the model is 



E 



^ ^ HjSj ^ ^ ®ij J ij SjSj 

i i<j 

^ Qijk(Jij SiSj + JjkSjSk + JikSiSk) 
i<j<k 

- ^ o,ijki(JijSiSj + JjkSjSk + JkiSkSi 

i<j<k<l 



• •) 

(21) 



Here we sum energies of spin clusters corresponding to 
motifs in the network. The second sum corresponds to 
simple edges. The third sum corresponds to triangles. 
The forth sum corresponds to motifs of size 4, and so on. 
The coupling Jjj determines the energy of interaction 
between spins i and j. Hi is a local magnetic field. In 
general case, the local fields Hi and the couplings Jij can 
be random quantities. 

In order to solve this model, we generalize the belief- 
propagation algorithm. At first, we remind the belief- 
propagation algorithm in application for the Ising model 
on tree-like uncorrelated complex networks without short 
loops (for more details, see Ref. @). In this case, there 
are only edges determined by the adjacency matrix aij 
and clustering coefficient is zero in the thermodynamic 
limit. Consider spin i. A nearest neighboring spin j sends 
to spin i a so-called message which we denote as fj,ji(Si). 
This message is normalized, 



(22) 



Within the belief-propagation algorithm, the probability 
that spin i is in a state Si is determined by the nor- 
malized product of incoming messages to spin i and the 
probabilistic factor e@ Hi * due to a local field Hf. 



MS^^je^H^). 
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(23) 



Here A is a normalization constant, j3 = 1/T is the re- 
ciprocal temperature. Now we can calculate the mean 



This equation shows that a parameter hji plays a role of 
an effective field produced by spin j at site i. Messages 
Hji(Si) obey a self-consistent equation which is called 
update rule, 



b y, e ~ mj: 

s,=±i 



n^i 



where the index n numerates nearest neighbors of vertex 
j, B is a normalization constant. The diagram repre- 
sentation of this equation is shown in Fig. [2] (a). The 
probabilistic factor exp[— j3E(j, i)] takes account of the 
interaction energy of spins j and i and a local field Hj , 



E(j, i) — HjSj (XjiJ jiSiS^ 



(28) 



Thus, a message from j to i is determined by the coupling 
between these spins and messages received by j from its 
neighbors except i. Multiplying Eq. (|2T[) by Si and sum- 
ming over Si = ±1, we obtain a self-consistent equation 
for the effective fields hji, 



tanh(/3/iji) = tanh(/3< 



jiJji 



tanh 



n(^i) 



(29) 

In the thermodynamic limit, this equation is exact for a 
tree-like graph. The Bethe-Peierls approach and the Bax- 
ter recurrence method give exactly the same result [j|[33j]. 
In numerical calculations, equations (|2T)) and may 
be solved by use of iterations. In a general case, an ana- 
lytical solution of these equations is unknown. In the case 
of all-to-all interactions with random couplings Jji (the 
Sherrington-Kirkpatrick model), this set of equations is 
reduced to well-known TAP equations [34| which are ex- 
act in the thermodynamic limit. These equations also 
give an exact solution of the ferromagnetic Ising model 
with uniform couplings Jji — J > on a random uncor- 
related graph with zero clustering coefficient d, |H, H|[ . 

Now we generalize the belief-propagation algorithm to 
the case of networks with given distributions of motifs 
described in Sec. |TTJ In a network which only consists 
of edges, a message goes along an edge from the spin at 
one edge end to the spin at the other edge end. For the 
networks with motifs, it is natural to introduce a mes- 
sage which is sent by a motif attached to a vertex. This 
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(a) , 



(b) , 




FIG. 2. Diagram representation of the belief-propagation up- 
date rules for messages from a motif to a destination vertex i. 
Arrows represent incoming messages, (a), (b) and (c) repre- 
sent update rules for an edge, a triangle, and a square. Math- 
ematical expressions of these diagrams are given by Eqs. (|27[) 
and p3]l . 

message goes to a spin to which this motif (hyperedge) is 
attached. Different motifs may be attached to a vertex, 
see Fig. [1] Let Mi{i) denote a cluster of size I attached 
to vertex i which we will call destination vertex. Vertices 
ji, j2, together with the destination vertex i form 

this motif. We introduce a message HM e (i)(^i) to spin i 
from a motif Mg(i). This message is normalized and can 
be written as follows, 

VM e (i)(Si) = exp(ph Me (i)Si^/[2cosh(/3h Me (i))]. (30) 

As above, the probability that spin i is in a state Si is 
determined by the normalized product of incoming mes- 
sages from motifs attached to spin i and the probabilistic 
factor e& HiSi , 

Pi(S i ) = ~e? H * s * J] »M t (i)(Si). (31) 

{Afc(i)} 

Using Eq. (f24|). we find that the mean moment (Si) is 
determined by effective fields h Ml ^ acting on spin i from 
motifs Mi(i), I — 2,3, 4, attached to i, see Fig. [TJ 

(Si) =tanh(/3tf 4 + /3 £ h MtW ). (32) 

|M/(i)} 



Here the sum is taken over motifs attached to i. 

Let us find an update rule for a message fJ-M e (i)(^i) 
from a given motif Mg(i) to vertex i. We introduce the 
following update rule: 

B £ e -^(M <( i)) -Q J| MMii(j)(5j ) = m(i) (^). 
{S J= ±1} j {M n ( 3 )=iM e (i)} 

(33) 

This rule shows that the message HM e (i)(^i) is equal to 
the product of incoming messages from motifs attached 
to all spins j in the motif Mi(i) except spin i and the 
motif Mi(i) itself (see Fig. [2] In Eq. (f33|) we average over 
all spin configurations of the spins Sj 1} Sj 2 , and Sj e _ 1 
in the motif. B is a normalization constant. E{Mp(i)) is 
an energy of the interaction between spins in the motif 
Mp(i). This update rule also takes account of local fields 
Hj acting on all spins except the destination spin i. We 
would like to note that equation (|33[) is valid for arbitrary 
motifs even with a complex internal structure. The only 
assumption is that a sparse network under consideration, 
in terms of hypergraphs, has a hypertree-like structure. 

For the sake of simplicity, let motifs M?(i) be finite 
loops of size I = 2, 3, . . . Then 

t-i e-i 
E(Me(i)) =-J2 H jn S jn - J2 Juu +1 S u S u+1 , (34) 

n— 1 n— 

where jo = je = i- If motifs are ^-cliques, then the energy 
E(Mi(i)) takes account of interactions between all spins 
in these cliques, 

E(M t (t)) = - Y HjSj - i Yl • / <-.- S >'... (35) 

Multiplying Eq. (|3"3"f by Si and summing over all spin 
configurations, we obtain an equation for the effective 
field h Mt (i), 

tanh(/3/i Mf(i) ) = {S l ) Mtil) . (36) 
The function on the right hand side is 

<*>"■"> = ^ S,e-^>K (37, 

v v " {S t ,S 31 ,...=±l} 

This function has a meaning of the mean moment of spin 
Si in the cluster Mi(i) with an energy 

e-i i 
E{MS)) = -£ H t (j n )S jn - J U, n+ iS jn S jn+1 . 

n=X n=0 

This energy takes into account both the interaction be- 
tween spins in this motif and total effective fields H t (j) 
acting on these spins. In turn, the field Ht(j) acting on 
j is the sum of a local field Hj and effective fields h,M m (j) 
produced by incoming messages from motifs M m (j) at- 
tached to vertex j except the motif Mi{i), see Fig. [3j 

H t {j)=Hj+ Y, h Mm{3)- ( 39 ) 



Z{Mg{i)) is the partition function of the cluster Mi(i), 
Z(Mt(i))= Yl e -W Mt <M. (40) 

Thus, the function (Si)M e (i) is a function 
F[HtVi),H t (j 2 ),...,H t Ut-i)] of the total fields 
Ht(j) acting on all spins in the motif Mf(i) except spin 
i. For a given network with motifs, it is necessary to 
solve Eq. (|36|) with respect to the effective fields h Ml ^y 
Then one can calculate the mean magnetic moments (Si) 
from Eq. (j3"2")l . Note that the only condition we used to 
derive the equations above was hypertree-like structure 
of the networks. The absence of correlations in these 
hypertree-like networks was not needed. Equations (|33[) 
and p6p are our main result which actually generalizes 
the Bethe-Peierls approach and the Baxter recurrence 
method. 

Let us study the ferromagnetic Ising model with uni- 
form couplings, Jji — J > 0, at zero magnetic field, i.e., 
Hi — H — 0, on a uncorrelated hypergraph which con- 
sists of edges and finite loops. In a random network, 
effective fields h Me ^ are also random variable and fluc- 
tuate from vertex to vertex. For a given £, we introduce 
the distribution function of fields hj^ t (i), 

1 N 

**( h ) = jE E 5{h~h Ml{%) ). (4i) 

i=l {U t {i)} 

Here, we sum over vertices i and attached motifs Mi(i) 
of size £. A — N(Q&) is the normalization constant. We 
assume that in the thermodynamic limit, N — > oo, the 
average over vertices in the network is equivalent to the 
average over the statistical network ensemble, Eq. (|T2"j). 
Using equation (I36[) . we obtain an equation for the dis- 
tribution function ^i(h), 

. t-i 
9 t (h)= / sfh-TtaxihT^M.^liMHtU^dHtij). 

J 3 = 1 

(42) 

Here (S) Mt is the function F[H t (l), H t (2), H t (£- 1)] 
defined by Eq. ([37]). H t (j) is a total field, Eq. (09]), acting 
on vertex j = 1, 2, . . . ,£ — 1 in the motif Mi, see Fig. [3] 
The integration is over fields H t (j) with the distribution 
function $e(H t (j)). Using Eq. (j3T)]) . we can find this 
function, 

Q2,Q 3 ,-.. We ' 

„ oo Q m Qe — 1 

m(jtt)ot=l a=l 

oo Q m 

m&l) a=l a=l 

(43) 
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FIG. 3. (a) A loop (motif Mi) of size £ is attached to des- 
tination vertex i. Wave arrows represent total effective fields 
Ht(j) acting on spins with index j — 1, 2, . . . , I— 1 in this loop, 
(b) In turn, the total effective field Ht(j) is a sum of effective 
fields produced by all motifs attached to vertex j except the 
motif Mi. There are Q2U) edges, Qs(j) triangles, and so on. 
However, the number of loops of size £ equals Qi(j) — 1, be- 
cause we should not account the motif Mi which is common 
for vertices j and i. 



Here P(Q 2 , Q3, ■■■) is given by Eq. ([T3]l. This is the prob- 
ability that a destination vertex has Q2 edges, Q3 trian- 
gles, and so on. In turn, incoming messages hyi m ip) 
to the destination spin from attached loops of size m 
are numerated by the index a, a = 1, 2, Q m . Only 
for incoming messages from loops of size £ we have 
a = 1,2, ...,Qi — 1, because we should not account for 
the motif Mg. The integration is over incoming messages 
to all vertices in the motif (hyperedge) Mi except the 
destination vertex i, see Fig. [3J Note that we have an 
additional factor Qi/(Qi), because in the configuration 
model Pi(Qi)Qi/(Qi) is the probability that if we arrive 
at a destination vertex along a hyperedge Mi, then this 
vertex has Qi — 1 outgoing hyperedges Mi. Equations 
(|4"2")l and (|43[) represent a set of self-consistent equa- 
tions for the functions ^i(h), £ = 2, 3, 4, ... . Then, using 
Eq. (f3"2"l) , one can find a distribution function of sponta- 
neous magnetic moments (Si) in the network. Equations 
(|4"2"| and (|4"3"| are also valid for networks with cliques. 
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IV. CRITICAL TEMPERATURE 

In the paramagnetic phase at zero magnetic held, there 
is no spontaneous magnetization and the effective fields 
are equal to zero, i.e., h,M c = 0. Therefore, we obtain 



(44) 



9 t (h) = S(h). 



for any I. This is the only solution of Eqs. (j42J) and (|43|) 
in the paramagnetic phase. Below a critical temperature 
T c , this solution becomes unstable and a non-trivial solu- 
tion for the function ^i(h) emerges. In this case, a mean 
value of the effective field h m, , 



h^ e (h)dh, 



(45) 



becomes non-zero. Here (...)t means the thermodynamic 
average. In the case of a continuous phase transition, 
(^m^)t tends to zero if temperature T tends to T c from 
below. Let use this fact. From Eq. (|36|) we obtain 



t-i 



f Unh(/3h)^ e (h)dh^ f (S} Mt Y[MHt(j))dH t (j), 

(46) 

We expand the functions tanh(/3/i) and (S)mi on the left 
and right hand sides of this equation in small h and Ht ( j), 
respectively: 



tanh(/3/i) = (3h + 0(h 3 ), 



e-i 



(Si) 



Mi: 



where we define 

d(Si)Mt 



Xe(ij) 



dH t (j) 



H t (j)=0 



J/M e 



H t (j)=0 



(47) 



(48) 



X£(ij) is a non-local susceptibility in a spin cluster Mi 
at zero magnetic field. Note that Txi{H) — 1- Assuming 
that at T — > T c — the first moment of the function e(h), 
i.e., (h,M m )T, is much larger than higher moments, i.e., 
{hu m )T 3> (^m )ti m the leading order we obtain a 
linear equation, 



(hM e )T 



Fe(T) 
(Qt) 



(Qt(Qi-i))(h Mt )T 



^ {QlQm){h Mm )T 



(49) 



where (...) defines an average over the distribution func- 
tion P(Q2,Q3,Qa, ■ • ■ ) given by Eq. The function 
Fg(T) is defined as follows, 



e-i 



F t (T)=T^2xe(ij) = T X e-l, 



(50) 



Xe is the total zero-field magnetic susceptibility of the 
Ising model on a ring of size £. Simple calculations give 



t. 



Ft{T) 



2t{l-t i - 1 ) 

(i-*)(i+^r 



£ > 3, 



(51) 



where t = tanh(J/T). In the paramagnetic phase, the 
set of linear equations (|49[) for parameters (/iMf)r only 
has a trivial solution, (/im { )t = 0. A non-trivial solution 
appears at a temperature at which 



detM = 0, 

where the matrix M mn is defined as follows: 
M mn = (Q m Q n ), m^n, 

{Qrn) 



(52) 



M mm = (Qm(Qm - 1)) - 



F m (T)' 



(53) 



for m,n — 2,3,.... Equation (|52[) determines the criti- 
cal point T c of the continuous phase transition. Below 
T c , spontaneous effective fields appear, i.e., (h,M e )T 0. 
Therefore, there is a non-zero spontaneous magnetic mo- 
ment. If all motifs are loops of equal length £, then the 
critical temperature T c is determined by the equation 



B t F e (T) = 1 



(54) 



where Bi is the average branching coefficient for these 
hyperedges [£- loops in this network), 



Bi = 



(QfXQi - 1)) 
(Qt) ' 



(55) 



In particular, if there are only edges, i.e., £ = 2, then Eq. 
(|5H) gives 



T c = 2J/ln( S2+l 



.5, 



(56) 



This result was found for uncorrelated random complex 
networks with arbitrary degree distributions [9j, [35J, l36| . 
Figure H] shows the dependence of the critical temper- 
ature T c versus the mean number of nearest neighbors 
(Q) = 2(Qi), Eq. (TP?) , in the networks having £-loop 
motifs, in which the Poissonian distribution of hyperde- 
grees is assumed. In this case Bi = Qg. One can see 
that, at a given (Q), the larger the loops the larger the 
critical temperature. At £ = 2 (an ordinary Erdos-Renyi 
graph), the Ising model has a lower T c in comparison to 
that at £ = 3. In the limit (Q) = 2{Q t ) > 1, we find that 



r c (/ = 2)/JS2(Q/) + o(l), 
T C {£>Z)/J = 2{Q l ) + l + o{\), 



(57) 



If there are only two motifs, for example, loops of size 
£ and £', then Eq. (|52p leads to the equation 



Bt 



1 



Fi(T) 



Bp 



(QtQe>) 2 
Fr(T)l ' (Qt)(Qt'Y 



(58) 
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FIG. 4. (Color online) Critical temperature T c of the fer- 
romagnetic Ising model versus the mean number of nearest 
neighbors (Q) = 2{Qi) in uncorrelated random networks with 
Poisson distribution of loops of size £ and the mean number 
(Qe) of these loops attached to a vertex. Note that the case 
£ — 2 is special: there are no loops, and we have the Erdos- 
Renyi graph. Inset displays T c versus the mean number (Qe) 
of ^-cliques (dashed lines), and, for comparison, T c for finite 
loops of the same size £ (solid lines). Here we put J = 1. 



From Eqs. (1511) and (I541 . it follows that if an uncorre- 
lated random hypergraph has a divergent second moment 
(Qm) f° r an Y 171 ^ 2, then T c becomes infinite in the infi- 
nite size limit. This result was obtained for ordinary un- 
correlated random complex network in Refs. 0, l35l l3(|. 
Moreover, because equations (|42|) and (|43|) have analyti- 
cal properties similar to ones studied in Refs. [9l.[35l[36|. 
we conclude that the ferromagnetic Ising model on con- 
sidered loopy networks has the same critical properties 
as ones on ordinary uncorrelated random networks. 

Another example of complex motifs are ^-cliques. Cal- 
culating the susceptibility \ °f a cluster of spins on this 
subgraph, we can find the function Fi(T) from Eq. (|50[) 
at £ > 3, 



F t (T) = {l-l) 



4It-2 (T) 



h{T) 



(59) 



It is not wonder that this critical temperature is larger 
than T c , Eq. ([57)1 . of a network consisting of short loops of 
size £ at a given mean degree (Qe). This result shows that 
the internal structure of motifs may influence strongly 
critical properties of the Ising model although that the 
percolation threshold may be the same (see below) . 



V. PERCOLATION THRESHOLD 

Equation (I52[) permits us to find the percolation 
threshold in these loopy networks. Below the percola- 
tion threshold, a network consists of finite clusters and 
there is no giant component. In this case, there is no 
phase transition in the Ising model. This spin system is 
in a paramagnetic state at any T. At a point of the birth 
of a giant component, there is a giant connected cluster 
and the critical temperature is T c = 0. Above the per- 
colation threshold, the critical temperature is non-zero, 
T c > 0. Therefore, in a general case for an arbitrary 
distribution function P(Q2, Q3, Q4, ■•■), the percolation 
point is determined by Eq. (p)2")) at T = 0. In this case, 
we have Fg(T = 0) = I — 1. For example, if there are 
only loops of size £ and £' then Eq. (|58[) takes a form, 



Be — 



1 



1 



1 



(" 



1 



(Qt 



(Qe)(Qt>) 



(62) 



In the case of £-loop motifs, equation gives the fol- 
lowing criterion for the birth of a giant connected com- 
ponent: 



DB, = L 



(63) 



Here we introduced the function 



where Bg is the average branching. At I = 2, this is the 
Molloy-Reed criterion for ordinary uncorrelated random 
networks (37j . The percolation threshold can be seen in 
Fig.@]as a critical value of the mean degree (Q), Eq. (fTTl) . 
below which T c is zero. Our results about the percolation 
threshold agree with results obtained in Refs. (22I I24I l38j 
by use of different approaches. It is interesting that we 
have Fi {T = 0) = £ — 1 for both finite loops and cliques 
of the same size £. Therefore, the percolation threshold 
Eq. (|63l) . i.e., the point of birth of the giant component, 
is also the same. 



VI. CONCLUSION 



I e (T) = Y,C e n exp\-(3J(2n-£) 



n=0 



where Cf> = nl/(n — £)\£\ is the binomial coefficient. If a 
network only consists of uncorrelated ^-cliques then the 
critical temperature of the ferromagnetic Ising model is 
given by Eq. (|5H) with the function Eq. (JSHJ) . In the case 
of the Poisson distribution of ^-cliques with the mean 
degree (Qe} ^> 1, we obtain an asymptotic result, 



(g0) In the present paper, we considered highly structured 

sparse networks with arbitrary distributions of motifs, 
which locally have hypertree-like structure. Using the 
configuration model for hypergraphs, we introduced a 
statistical ensemble of these random networks and found 
the probability of the realization of the network with a 
given sequence of edges, finite loops, and cliques. We 
generalized the belief-propagation algorithm to networks 
with arbitrary distributions of motifs. Using this algo- 
rithm, we solved the Ising model on the networks with ar- 
(61) bitrary distributions of finite loops and cliques. We found 
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an exact critical temperature of the ferromagnetic Ising 
model with uniform coupling between spins. We demon- 
strated that clustering increases the critical temperature 
in comparison with an ordinary tree- like network with the 
same mean degree. We showed that networks with loops 
of large size have a larger critical temperature in compar- 
ison with networks having loops of smaller size. Our so- 
lution also enabled us to find the birth point of the giant 
connected component in sparse networks with arbitrary 
distributions of finite loops and cliques in agreement with 
Refs. dHHii]. We believe that the proposed general- 
ized belief propagation algorithm may be used for study- 



ing dynamical processes and variety of models on highly 
structured networks with complex motifs. 
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